On the adoption of anomaly detection for packed executable filtering

Malware packing is a common technique employed to hide malicious code and to avoid static analysis. In order to fully inspect the contents of the executable, unpacking techniques must be applied. Unfortunately, generic unpacking is computationally expensive. For this reason, it is important to filter binaries in order to correctly handle them. In previous work, we proposed the adoption of anomaly detection for the classification of packed and not packed binaries using features based on the Portable Executable structure. In this paper, we extend this work and thoroughly evaluate the method with a different dataset and two different feature sets, rendering new conclusions. While anomaly detection is reaffirmed as a sound method for the discrimination of packed and not packed binaries, Portable Executable structure based features present limitations to distinguish custom packed files from not packed files. a 2014 Elsevier Ltd. All rights reserved.